Skip to content

fix: don't drop items when close races with the final put!#6

Merged
zsoerenm merged 1 commit into
mainfrom
fix-iterate-close-race
Jul 13, 2026
Merged

fix: don't drop items when close races with the final put!#6
zsoerenm merged 1 commit into
mainfrom
fix-iterate-close-race

Conversation

@zsoerenm

Copy link
Copy Markdown
Member

Problem

The consumer-side drain checks (take!, take!(ch, output), wait, and iterate) read the head/tail indices, observed an empty buffer, and then separately read the closed flag.

Although close() is ordered after every put!, the empty-snapshot read and the closed read are distinct atomic operations. So a producer's final put! can become visible to the consumer only after its close():

  1. consumer reads head → sees empty (old value)
  2. producer advances head (publishes item), then sets closed = true
  3. consumer reads closed → sees true
  4. consumer concludes closed-and-empty → throws / ends iteration → drops the last item

This surfaces as a rare (~0.1%) data-loss flake under load — e.g. a tee/pipeline consumer receiving fewer items than were produced.

Fix

Re-read the indices after observing closed. Because close() is ordered after every put!, a re-read at that point observes all published items, so a throw only happens when the channel is genuinely drained:

if ch.closed[] && tail == ch.head[]   # re-read head after seeing closed
    check_closed_and_throw(ch)
end

Applied to take!, take!(ch, output), and wait. Also simplified iterate to always attempt the (now race-free) take! rather than guarding it with the racy isopen(ch) || isready(ch) snapshot — take! already drains before throwing and check_closed_and_throw propagates bound-task exceptions.

Testing

Added a threaded regression test: produce N items, close(), then iterate and assert all N are received, over many trials. It fails reliably on the old code and passes on the fixed code. Verified 0 failures over 60,000 additional stress trials (iterate + take! paths) locally with --threads=4.

🤖 Generated with Claude Code

The consumer-side drain checks in take!, take!(ch, output), wait and iterate
read the head/tail indices, observed an empty buffer, and *then* separately
read the closed flag. Because close() is ordered after every put! but the
empty-snapshot and closed reads are distinct operations, a producer's final
put! could become visible to the consumer only after its close(): the consumer
would observe the channel as closed-and-empty and throw / end iteration,
silently dropping the last item(s).

Fix the drain checks to re-read the indices after observing closed: since
close() is ordered after every put!, a re-read then observes all published
items, so throwing only happens when the channel is genuinely drained.

Also simplify iterate to always attempt the (now race-free) take! instead of
guarding it with the racy `isopen(ch) || isready(ch)` snapshot; take! already
drains before throwing and propagates bound-task exceptions.

Add a threaded regression test that produces N items then closes and asserts
the consumer iterates all N, over many trials.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@codecov

codecov Bot commented Jul 13, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 99.27%. Comparing base (497c946) to head (4bfe236).

Additional details and impacted files
@@            Coverage Diff             @@
##             main       #6      +/-   ##
==========================================
+ Coverage   94.48%   99.27%   +4.78%     
==========================================
  Files           1        1              
  Lines         145      137       -8     
==========================================
- Hits          137      136       -1     
+ Misses          8        1       -7     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@github-actions

Copy link
Copy Markdown

Benchmark Results (Julia v1)

Time benchmarks
main 4bfe236... main / 4bfe236...
batch/batch=128 27.7 ± 9.4 μs 28 ± 8.2 μs 0.988 ± 0.44
batch/batch=16 0.0425 ± 0.0099 ms 0.0429 ± 0.0098 ms 0.989 ± 0.32
batch/batch=256 27.6 ± 3.1 μs 31.6 ± 10 μs 0.873 ± 0.3
batch/batch=32 0.0338 ± 0.00076 ms 0.0338 ± 0.01 ms 1 ± 0.3
batch/batch=64 29.3 ± 0.4 μs 0.038 ± 0.0013 ms 0.771 ± 0.028
batch/batch=8 0.0635 ± 0.01 ms 0.06 ± 0.0011 ms 1.06 ± 0.17
batch/put!_batch=128 0.16 ± 0.091 μs 0.1 ± 0.08 μs 1.6 ± 1.6
batch/put!_batch=16 0.04 ± 0 μs 0.04 ± 0 μs 1 ± 0
batch/put!_batch=256 0.14 ± 0.001 μs 0.14 ± 0.01 μs 1 ± 0.072
batch/put!_batch=32 0.04 ± 0.01 μs 0.04 ± 0.01 μs 1 ± 0.35
batch/put!_batch=64 0.07 ± 0.091 μs 0.07 ± 0.09 μs 1 ± 1.8
batch/put!_batch=8 0.04 ± 0.01 μs 0.04 ± 0.01 μs 1 ± 0.35
batch/single 0.158 ± 0.0026 ms 0.177 ± 0.028 ms 0.893 ± 0.14
batch/take!_batch=128 0.1 ± 0.01 μs 0.1 ± 0.01 μs 1 ± 0.14
batch/take!_batch=16 0.05 ± 0.001 μs 0.05 ± 0.001 μs 1 ± 0.028
batch/take!_batch=256 0.321 ± 0.13 μs 0.331 ± 0.13 μs 0.968 ± 0.55
batch/take!_batch=32 0.06 ± 0.01 μs 0.06 ± 0 μs 1 ± 0.17
batch/take!_batch=64 0.08 ± 0.01 μs 0.08 ± 0.01 μs 1 ± 0.18
batch/take!_batch=8 0.05 ± 0 μs 0.05 ± 0 μs 1 ± 0
batch/take!_buffer_batch=128 0.08 ± 0.01 μs 0.081 ± 0.01 μs 0.988 ± 0.17
batch/take!_buffer_batch=16 0.04 ± 0.009 μs 0.04 ± 0 μs 1 ± 0.23
batch/take!_buffer_batch=256 0.14 ± 0.01 μs 0.14 ± 0.019 μs 1 ± 0.15
batch/take!_buffer_batch=32 0.04 ± 0.01 μs 0.04 ± 0.01 μs 1 ± 0.35
batch/take!_buffer_batch=64 0.06 ± 0 μs 0.06 ± 0 μs 1 ± 0
batch/take!_buffer_batch=8 0.04 ± 0.01 μs 0.04 ± 0.01 μs 1 ± 0.35
channel/buffer=1 5.01 ± 0.04 ms 5.21 ± 0.069 ms 0.963 ± 0.015
channel/buffer=1024 0.663 ± 0.02 ms 0.639 ± 0.017 ms 1.04 ± 0.041
channel/buffer=16 0.932 ± 0.014 ms 0.933 ± 0.0087 ms 1 ± 0.018
channel/buffer=256 0.644 ± 0.013 ms 0.646 ± 0.013 ms 0.997 ± 0.029
channel/buffer=4 1.79 ± 0.017 ms 1.8 ± 0.022 ms 0.994 ± 0.015
channel/buffer=64 0.703 ± 0.013 ms 0.713 ± 0.011 ms 0.987 ± 0.024
channel/put! 0.08 ± 0.01 μs 0.08 ± 0.01 μs 1 ± 0.18
channel/take! 0.05 ± 0.01 μs 0.05 ± 0.01 μs 1 ± 0.28
pipechannel/buffer=1 3.79 ± 0.036 ms 3.76 ± 0.022 ms 1.01 ± 0.011
pipechannel/buffer=1024 0.158 ± 0.0097 ms 0.151 ± 0.0095 ms 1.05 ± 0.092
pipechannel/buffer=16 0.393 ± 0.012 ms 0.388 ± 0.012 ms 1.01 ± 0.044
pipechannel/buffer=256 0.179 ± 0.0093 ms 0.177 ± 0.0033 ms 1.01 ± 0.056
pipechannel/buffer=4 1.06 ± 0.008 ms 1.06 ± 0.0092 ms 1.01 ± 0.012
pipechannel/buffer=64 0.23 ± 0.0096 ms 0.212 ± 0.0078 ms 1.08 ± 0.06
pipechannel/put! 30 ± 1 ns 30 ± 1 ns 1 ± 0.047
pipechannel/take! 30 ± 1 ns 30 ± 1 ns 1 ± 0.047
time_to_load 0.0338 ± 0.0039 s 0.0332 ± 0.00034 s 1.02 ± 0.12
Memory benchmarks
main 4bfe236... main / 4bfe236...
batch/batch=128 15 allocs: 2.91 kB 15 allocs: 2.91 kB 1
batch/batch=16 15 allocs: 1.09 kB 15 allocs: 1.09 kB 1
batch/batch=256 17 allocs: 4.86 kB 17 allocs: 4.86 kB 1
batch/batch=32 15 allocs: 1.34 kB 15 allocs: 1.34 kB 1
batch/batch=64 15 allocs: 1.84 kB 15 allocs: 1.84 kB 1
batch/batch=8 15 allocs: 0.969 kB 15 allocs: 0.969 kB 1
batch/put!_batch=128 0 allocs: 0 B 0 allocs: 0 B
batch/put!_batch=16 0 allocs: 0 B 0 allocs: 0 B
batch/put!_batch=256 0 allocs: 0 B 0 allocs: 0 B
batch/put!_batch=32 0 allocs: 0 B 0 allocs: 0 B
batch/put!_batch=64 0 allocs: 0 B 0 allocs: 0 B
batch/put!_batch=8 0 allocs: 0 B 0 allocs: 0 B
batch/single 11 allocs: 0.703 kB 13 allocs: 0.812 kB 0.865
batch/take!_batch=128 2 allocs: 1.09 kB 2 allocs: 1.09 kB 1
batch/take!_batch=16 2 allocs: 0.188 kB 2 allocs: 0.188 kB 1
batch/take!_batch=256 3 allocs: 2.07 kB 3 allocs: 2.07 kB 1
batch/take!_batch=32 2 allocs: 0.312 kB 2 allocs: 0.312 kB 1
batch/take!_batch=64 2 allocs: 0.562 kB 2 allocs: 0.562 kB 1
batch/take!_batch=8 2 allocs: 0.125 kB 2 allocs: 0.125 kB 1
batch/take!_buffer_batch=128 0 allocs: 0 B 0 allocs: 0 B
batch/take!_buffer_batch=16 0 allocs: 0 B 0 allocs: 0 B
batch/take!_buffer_batch=256 0 allocs: 0 B 0 allocs: 0 B
batch/take!_buffer_batch=32 0 allocs: 0 B 0 allocs: 0 B
batch/take!_buffer_batch=64 0 allocs: 0 B 0 allocs: 0 B
batch/take!_buffer_batch=8 0 allocs: 0 B 0 allocs: 0 B
channel/buffer=1 12 allocs: 0.75 kB 12 allocs: 0.75 kB 1
channel/buffer=1024 19 allocs: 17.2 kB 19 allocs: 17.2 kB 1
channel/buffer=16 14 allocs: 1.14 kB 14 allocs: 1.14 kB 1
channel/buffer=256 17 allocs: 6.51 kB 17 allocs: 6.51 kB 1
channel/buffer=4 13 allocs: 0.844 kB 13 allocs: 0.844 kB 1
channel/buffer=64 15 allocs: 2.28 kB 15 allocs: 2.28 kB 1
channel/put! 1 allocs: 0.0469 kB 1 allocs: 0.0469 kB 1
channel/take! 0 allocs: 0 B 0 allocs: 0 B
pipechannel/buffer=1 11 allocs: 0.703 kB 13 allocs: 0.812 kB 0.865
pipechannel/buffer=1024 11 allocs: 0.703 kB 13 allocs: 0.812 kB 0.865
pipechannel/buffer=16 11 allocs: 0.703 kB 13 allocs: 0.812 kB 0.865
pipechannel/buffer=256 11 allocs: 0.703 kB 13 allocs: 0.812 kB 0.865
pipechannel/buffer=4 11 allocs: 0.703 kB 13 allocs: 0.812 kB 0.865
pipechannel/buffer=64 11 allocs: 0.703 kB 13 allocs: 0.812 kB 0.865
pipechannel/put! 0 allocs: 0 B 0 allocs: 0 B
pipechannel/take! 0 allocs: 0 B 0 allocs: 0 B
time_to_load 0.145 k allocs: 11 kB 0.145 k allocs: 11 kB 1

@zsoerenm
zsoerenm merged commit acee8ae into main Jul 13, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant